Papers with negative sampling strategies
Dense Hierarchical Retrieval for Open-domain Question Answering (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Dense neural text retrieval has achieved promising results on open-domain Question Answering (QA) current dense retrievers require splitting documents into short passages that usually contain local, partial and sometimes biased context, and may yield inaccurate and misleading hidden representations, thus deteriorating the final retrieval result. |
| Approach: | They propose a hierarchical framework which can generate accurate dense representations of passages by utilizing both macroscopic semantics in the document and microscopic specific to each passage. |
| Outcome: | The proposed framework significantly outperforms the original dense passage retriever and helps an end-to-end QA system outperfect the strong baselines on multiple open-domain QA benchmarks. |
Assessing In-context Learning and Fine-tuning for Topic Classification of German Web Data (2024.acl-srw)
Copied to clipboard
| Challenge: | Using a few hundred annotated data points per topic, we detect content related to three German policies in a database of scraped webpages. |
| Approach: | They propose to use annotated data to train a binary classification task to detect topic-related content in a scraped database of webpages. |
| Outcome: | The proposed model detects content related to three German policies in a scraped database of scrapes of webpages using a few hundred annotated data points per topic. |
SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval (2022.emnlp-industry)
Copied to clipboard
Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-rong Wen, Nan Duan
| Challenge: | Existing methods for sapping negatives from large document pool suffer from the uninformative or false negative problem. |
| Approach: | They propose a method to sample negatives from a large document pool using a new sampling probability distribution. |
| Outcome: | The proposed method can be used to sample more ambiguous negatives on four public and one industry datasets. |
RAG4ITOps: A Supervised Fine-Tunable and Comprehensive RAG Framework for IT Operations and Maintenance (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have improved the open-domain QA’s performance, but how to efficiently handle enterprise-exclusive corpora and build domain-specific QA systems are still not studied for industrial applications. |
| Approach: | They propose a general and comprehensive framework based on Retrieval Augmented Generation (RAG) and facilitate the whole business process of establishing QA systems for IT operations and maintenance. |
| Outcome: | The proposed framework achieves superior results on two kinds of QA tasks. |
DORE: Document Ordered Relation Extraction based on Generative Framework (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing generative methods do not fit document-level relation extraction tasks where there are multiple entities and relational facts. |
| Approach: | They propose to generate a symbolic and ordered sequence from relation matrix which is easier to learn and introduce several negative sampling strategies to improve the performance with balanced signals. |
| Outcome: | The proposed method can improve the performance of the generative DocRE models on four datasets. |
ECtHR-PCR: A Dataset for Precedent Understanding and Prior Case Retrieval in the European Court of Human Rights (2024.lrec-main)
Copied to clipboard
| Challenge: | Prior case retrieval datasets do not simulate a realistic setting because they use complete case documents while only masking references to prior cases. |
| Approach: | They propose a prior case retrieval dataset based on judgements from the European Court of Human Rights which explicitly separate facts from arguments and exhibit precedential practices. |
| Outcome: | The proposed datasets do not simulate a realistic setting and expose queries to spurious patterns left behind by citation masks, potentially short-circuiting a comprehensive understanding of case facts and legal principles. |
Multi-stage Training with Improved Negative Contrast for Neural Passage Retrieval (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural firststage retrieval models overcome lexical gap issue by projecting query and document to a shared dense space. |
| Approach: | They propose a multi-stage framework for neural passage retrieval using synthetic data, negative sampling, and fusion techniques. |
| Outcome: | The proposed framework improves retrieval accuracy and enhances the negative contrast in both stages. |